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This action is responsive to the amendment and remarks file on 4/23/08. 
Claims 1-21 are presented for further examination. 

DETAILED ACTION 
Response to Arguments 
Applicant's arguments filed 4/17/08 have been fiiUy considered but they are not 
persuasive. 

Applicant argues that the cited art of record (Lundblad et al) did not disclose "estimate 
the time needed to deliver the first information" and "estimated the time needed to deliver the 
second information" 

Examiner disagrees. 

Lundblad discloses i) estimating total time needed to deliver the first information (a) to 
the first output means or (b) to a store local to the first output means (column 1, lines 55-58); 
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ii) estimating total time needed to deliver the second information (a) to the second output means 
or (b) to a store local to the second output means (column 1, lines 58-59); 

The processing time, here in Lundblad, means that the determine/estimate the time it 
takes to deliver the information onto the output means (timestamp, to determine/estimate time 
the total time needed to deliver the message to the display, which is also being supported in Fig 6 
and more detail explanation located in Col 7, Lines 14-65). 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be enlillcd lo a palcnl unless 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

Claims 1 -4, 6-10, 1 6, 20 and 21 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Lundblad et al. hereinafter Lundblad (US 6,906,755 B2). 

1 . Regarding claim 1 , Lundblad teaches: 

A method of synchronizing the delivery of (a) first information which is to be presented to a user 
via first output means of a multi-modal interface (b) second information which is to be 
presented to the user via second output means of the multi-modal interface, the method 
comprising: 
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i) estimating total time needed to deliver the first information (a) to the first output means 
or (b) to a store local to the first output means (column 1, lines 55-58); 

ii) estimating total time needed to deliver the second information (a) to the second output 
means or (b) to a store local to the second output means (column 1, lines 58-59); and 

iii) using the estimates obtained in step i) or step ii) to determine whether presentation to 
the user of the first or second information needs to be delayed to achieve a desired synchronism 
of presentation (column 1, lines 59-62); and 

iv) applying any delay determined in step iii) to achieved the desired synchronism of 
presentation (column 1, lines 62-65). 

2. Regarding claim 2, Lundblad teaches: 

A method as claimed in claim 1, wherein the first and second ou^ut means are provided by a 
single output device (figure 2, element 108 - TV). 

3. Regarding claim 3, Lundblad teaches: 

A method as claimed in claim 1, wherein at least one of the first and second output means is a 
visual display means (figure 2, TV screen for display of video). 

4. Regarding claim 4, Lundblad teaches: 

A method as claimed in claim 1, wherein at least one of the first and second output means is an 
audio reproduction means (figure 1, TV speaker for audio). 
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5 . Regarding claim 6, Lundblad teaches: 

A method as claimed in claim 1, wherein the first means is a visual display means (figure 1, 
video) and the second means is an audio reproduction means (figure 1, audio). 

6. Regarding claim 7, Lundblad teaches: 

A method of synchronizing delivery of (a) first information which is to be presented to the user 
via a visual display of a multi-modal interface and (b) second information which is to be 
presented to the user over a visual or an audio interface of the multi-modal interface, the method 
comprising: 

i) estimating total time needed to deliver the first information ~ to the visual display or 
(b) to a store local to the visual display (column 1, lines 55-58); 

ii) estimating total time needed to deliver the second information (a) to the visual or 
audio interface or (b) to a store local to the visual or audio interface (column 1, lines 58-59); and 

iii) using the estimates obtained in step i) or step ii) to determine whether presentation of 
the first or second information to the user needs to be delayed to achieve a desired synchronism 
of presentation (column 1, lines 59-62); and 

vi) applying delay determined in step iii) to achieved the desired synchronism of 
presentation (colximn 1, lines 62-65). 

7. Regarding claim 8, Lundblad teaches: 

A method of synchronizing the delivery (a) of first information which is to be presented to the 
user via a visual display of a multi-modal interface and (b) of second information which is to be 
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presented to the user over an audio interface of the multi- modal interface, the method 
comprising: 

i) estimating total time needed to deliver the first information (a) to the visual display or 
(b) to a store local to the visual display (column 1, liens 55-58); 

ii) estimating total time needed to deliver the second information (a) to the audio 
interface (b) or to a store local to the audio interface (column 1, lines 58-59); and 

iii) if the total time estimated in step i) is more than that estimated in step ii) delaying 
presentation of the second information to the user sufficiently to enable the first information to 
be presented to the user before the second information is presented to the user (column 1, lines 
58-62; figure 5, elements 510 and 514). 

8. Regarding claim 9, Lundblad teaches: 

A method as claimed in claim 7, wherein the delivery of the first information in step (i) is 
controlled by a server process, delivery of the first information involving delivery of that 
information to a client of the server process (figure 1, element 102 - content server). 

9. Regarding claim 10, Lundblad teaches: 

A method as claimed in claim 7, wherein the delivery of the second information in step (ii) is 
controlled by a server process, delivery of the second information involving delivery of that 
information to a client of the server process (figure 1, element 102 - content server). 



10. Regarding claim 16, Lundblad teaches: 
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A method as claimed in claim 7, wherein the estimate of total time produced in step i) includes a 
component for the time taken to render the first information on the visual display (column 6, 
lines 8-11, "presentation delays"; column 7, lines 56-59). 

1 1 . Regarding claim 20, Lundblad teaches: 

A system of apparatus for the delivery (a) of first information which is to be presented to the user 
via a visual display of a multi-modal interface and (b) of second information which is to be 
presented to the user over a visual or an audio interface of the multi-modal interface, the system 
including processing means configured to: 

estimate total time needed to deliver the first Information to the visual display or to a 
store local to the visual display (column 1, lines 55-58); 

estimate total time needed to deliver the second information to the visual or audio 
interface or to a store local to the visual or audio interface (column 1, lines 58-59); and 

to use the estimates thereby obtained to determine whether-t-he presentation to the user 
of the first or second information to the user needs to be delayed to achieve a desired 
synchronism of presentation (column 1, lines 59-62), and 

to cause any delay determined to be necessary to be applied to achieve the desired 
synchronism of presentation (coliimn 1, lines 62-65). 

1 2 . Regarding claim 2 1 , Lundblad teaches : 

A system of apparatus for the delivery (a) of first information which is to be presented to the user 
via first output means of a multi-modal interface and (b) of second information which is to be 
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presented to the user via second output means of the multi-modal interface, the system including 
processing means configured to: 

estimate the total time needed to deliver the first information to the first output means or 
to a store local to the first output means (column 1, lines 55-58); 

estimate the total time needed to deliver the second information to second output means 
or to a store local to the second output means (column 1, lines 58-59); and 

to use the estimates thereby obtained to determine whether presentation of the first or 
second information to the user needs to be delayed to achieve a desired synchronism of 
presentation (column 1, lines 59-62); and 

to cause any delay determined to be necessary to be applied to achieve the desired 
synchronism of presentation (column 1, lines 62-65). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lundblad as 
applied to claim 1 above, and fiirther in view of Dutta et al.. Patent No.: US 6,961,458 B2 
(hereinafter Dutta). 



13. 



Referring to Claim 5, Lundblad teaches the limitation of claim 1 for the reasons above. 
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Lundblad does not explicitly disclose that the output means are tactile reproduction 

means. 

Dutta teaches a data output mean for visually impaired user. Column 6, lines 1-3 teach 
the output device that generates tactile output like Braille. Dutta further provide the advantage of 
using tactile to communicate information to the visually impaired user. 

It would have been obvious to one of ordinary skill in the art, having the teaching of 
Lundblad and Dutta before them at the time the invention was made to modify the system of 
Charlton et al. to include tactile output means as taught by Dutta 

One of ordinary skill in the art would have been motivated to make this modification in 
order to allow a visually impaired user to receive data in a communication system. Dutta teaches 
that transmitting and receiving information in a communication system has become culture 
fixture and that visually impaired user relies on tools like tactile to help them be part of that 
culture. 

14. Claims 1 1-15 and 17-19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Limdblad as applied to claim 7 above, and fiirther in view of Grove et al. Patent No.: US 
6,820,133 Bl (hereinafter Grove). 
Regarding claims 11 and 12, 

Lundblad teaches the limitation of claim 7 for the reasons above. Lundblad teaches the 
determination of processing delay includes presentation delay. 

Lundblad does not explicitly disclose latency of a communication channel. 
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Grove teaches measuring of latency of a data transmission from one node to another 
(Column 15 lines 27-37). Lundblad fiirther teach utilizing the measured latency to estimate the 
communication time between two devices (column 18, lines 53-62). 

Lundblad suggests the utilization of time related to transmission of data in the 
presentation delay. It would have been obvious to one of ordinary skill in the art, having the 
teachings of Lundblad and Grove before them at the time the invention was made to modify the 
system and method of Lundblad to use measured latency for estimation of data transmission time 
as taught by Grove. 

One of ordinary skill in the art would have been motivated to make this modification in 
order to better estimate the processing delay needed. 

15. Regarding claim 13, Lundblad in view of Grove teach the method and system according 
to claim 1 1, as described above. 

Grove further teaches that the measurement of latency involves a server process sending 
a communication to an associated client to ehcit a response therefrom, the measurement of 
latency being derived from the duration of the interval between the sending of the 
communication and the receipt of the response (column 15, lines 29-32). 

16. Regarding claims 14 and 15, 

Lundblad teaches the limitation of claim 7 for the reasons above. Lundblad ftirther teach 
that knowledge of the quantity of video (first information) and audio (second information) are 
used to estimate the processing delay (column 5, lines 12-14 and column 6, lines 45-62). 
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Lundblad does not explicitly disclose the bandwidth information of the communication 
channel is used to calculate the time required to transmit the information and subsequently used 
in the estimation of the processing delay. 

Grove teaches using bandwidth figures to estimate the communication time between two 
devices (column 18, lines 53-62). 

It would have been obvious to one of ordinary skill in the art, having the teachings of 
Lundblad and Grove before them at the time the invention was made to modify the system and 
method of Lundblad to use bandwidth information for estimation of the data transmission time as 
taught by Grove. 

One of ordinary skill in the art would have been motivated to make this modification in 
order to better estimate the processing delay needed. 

17. Claim 17 is rejected on the same basis as claims 11-12 and 14-15. See the discussions 
regarding claims 11-12 and 14-15 above for details of this disclosure. 

18. Claim 18 is rejected on the same basis as claims 1 1 and 12. See the discussions regarding 
claims 1 1 and 12 above for details of this disclosure. 

19. Claim 19 is rejected on the same basis as claims 14 and 15. See the discussions regarding 
claims 14 and 15 above for details of this disclosure. 



Conclusion 
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Examiner's Notes: Examiner has cited particular columns and line numbers in the 
references applied to the claims above for the convenience of the apphcant. Although the 
specified citations are representative of the teachings of the art and are applied to specific 
limitations within the individual claim, other passages and figures may apply as well. It is 
respectfully requested from the applicant in preparing responses, to fully consider the references 
in entirety as potentially teaching all or part of the claimed invention, as well as the context of 
the passage as taught by the prior art or disclosed by the Examiner. In the case of amending the 
claimed invention. Applicant is respectfiiUy requested to indicate the portion(s) of the 
specification which dictate(s) the structure relied on for proper interpretation and also to verify 
and ascertain the metes and bounds of the claimed invention. 

Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Apphcant is 
reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Karen C. Tang whose telephone number is (571)272-31 16. The 
examiner can normally be reached on M-F 7-3. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Follansbee can be reached on (571)272-3964. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Elecfronic Business Center (EBC) at 866-217-9197 (toll-free). 



/K. C. 1.1 

Examiner, Art Unit 2151 



/John Follansbee/ 

Supervisory Patent Examiner, Art Unit 2151 



